视频修复旨在从多个低质量框架中恢复多个高质量的帧。现有的视频修复方法通常属于两种极端情况，即它们并行恢复所有帧，或者以复发方式恢复视频框架，这将导致不同的优点和缺点。通常，前者具有时间信息融合的优势。但是，它遭受了较大的模型尺寸和密集的内存消耗；后者的模型大小相对较小，因为它在跨帧中共享参数。但是，它缺乏远程依赖建模能力和并行性。在本文中，我们试图通过提出经常性视频恢复变压器（即RVRT）来整合两种情况的优势。 RVRT在全球经常性的框架内并行处理本地相邻框架，该框架可以在模型大小，有效性和效率之间实现良好的权衡。具体而言，RVRT将视频分为多个剪辑，并使用先前推断的剪辑功能来估计后续剪辑功能。在每个剪辑中，通过隐式特征聚合共同更新不同的帧功能。在不同的剪辑中，引导的变形注意力是为剪辑对齐对齐的，该剪辑对齐可预测整个推断的夹子中的多个相关位置，并通过注意机制汇总其特征。关于视频超分辨率，DeBlurring和DeNoising的广泛实验表明，所提出的RVRT在具有平衡模型大小，测试内存和运行时的基准数据集上实现了最先进的性能。
translated by 谷歌翻译
鉴于近期对视觉描述符的隐私开启的关于场景启示符的分析，我们开发隐藏输入图像内容的描述符。特别是，我们提出了对培训防止图像重建的视觉描述符的对抗性学习框架，同时保持匹配精度。我们允许一个特征编码网络和图像重建网络彼此竞争，使得特征编码器尝试利用其生成的描述符推出图像重建，而重构器尝试从描述符恢复输入图像。实验结果表明，通过我们的方法获得的视觉描述符显着恶化了对应匹配和相机定位性能的最小影响。
translated by 谷歌翻译
translated by 谷歌翻译
The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a subnetwork specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.
translated by 谷歌翻译
Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-called FlowNet was enabled by a large synthetically generated dataset. The present paper extends the concept of optical flow estimation via convolutional networks to disparity and scene flow estimation. To this end, we propose three synthetic stereo video datasets with sufficient realism, variation, and size to successfully train large networks. Our datasets are the first large-scale datasets to enable training and evaluating scene flow methods. Besides the datasets, we present a convolutional network for real-time disparity estimation that provides state-of-the-art results. By combining a flow and disparity estimation network and training it jointly, we demonstrate the first scene flow estimation with a convolutional network.
translated by 谷歌翻译
Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the tasks where CNNs were successful. In this paper we construct appropriate CNNs which are capable of solving the optical flow estimation problem as a supervised learning task. We propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations.Since existing ground truth datasets are not sufficiently large to train a CNN, we generate a synthetic Flying Chairs dataset. We show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI, achieving competitive accuracy at frame rates of 5 to 10 fps.
translated by 谷歌翻译
We study the multiclass classification problem where the features come from the mixture of time-homogeneous diffusions. Specifically, the classes are discriminated by their drift functions while the diffusion coefficient is common to all classes and unknown. In this framework, we build a plug-in classifier which relies on nonparametric estimators of the drift and diffusion functions. We first establish the consistency of our classification procedure under mild assumptions and then provide rates of cnvergence under different set of assumptions. Finally, a numerical study supports our theoretical findings.
translated by 谷歌翻译
Given a dataset of expert agent interactions with an environment of interest, a viable method to extract an effective agent policy is to estimate the maximum likelihood policy indicated by this data. This approach is commonly referred to as behavioral cloning (BC). In this work, we describe a key disadvantage of BC that arises due to the maximum likelihood objective function; namely that BC is mean-seeking with respect to the state-conditional expert action distribution when the learner's policy is represented with a Gaussian. To address this issue, we introduce a modified version of BC, Adversarial Behavioral Cloning (ABC), that exhibits mode-seeking behavior by incorporating elements of GAN (generative adversarial network) training. We evaluate ABC on toy domains and a domain based on Hopper from the DeepMind Control suite, and show that it outperforms standard BC by being mode-seeking in nature.
translated by 谷歌翻译
当应用于自动驾驶汽车设置时，行动识别可以帮助丰富环境模型对世界的理解并改善未来行动的计划。为了改善自动驾驶汽车决策，我们在这项工作中提出了一种新型的两阶段在线行动识别系统，称为RADAC。RADAC提出了主动剂检测的问题，并在直接的两阶段管道中以进行动作检测和分类的直接识别人类活动识别中的参与者关系的想法。我们表明，我们提出的计划可以胜过ICCV2021 ROAD挑战数据集上的基线，并通过将其部署在真实的车辆平台上，我们演示了对环境中代理行动的高阶理解如何可以改善对真实自动驾驶汽车的决策。
translated by 谷歌翻译
近期量子系统嘈杂。串扰噪声已被确定为超导噪声中间尺度量子（NISQ）设备的主要噪声来源之一。串扰源于附近Qubits上的两Q量门门的并发执行，例如\ texttt {cx}。与单独运行相比，它可能会大大提高门的错误率。可以通过调度或硬件调整来减轻串扰。然而，先前的研究在汇编的后期很晚，通常是在完成硬件映射之后的。它可能会错过优化算法逻辑，路由和串扰的巨大机会。在本文中，我们通过在早期编译阶段同时考虑所有这些因素来推动信封。我们提出了一个称为CQC的串扰感知量子程序汇编框架，该框架可以增强串扰缓解，同时实现令人满意的电路深度。此外，我们确定了从中间表示向电路转换的机会，例如，以特定的特定串扰缓解措施，例如，\ texttt {cx}梯子构造在变异的量子eigensolvers（VQE）中。通过模拟和Real IBM-Q设备进行评估表明，我们的框架可以显着将错误率降低6 $ \ times $，而与最先进的门调度相比，仅$ \ sim $ 60 \％\％的电路深度方法。特别是对于VQE，我们使用IBMQ Guadalupe证明了49 \％的回路深度减少，而对H4分子的先前ART进行了9.6 \％的保真度改善。我们的CQC框架将在GitHub上发布。
translated by 谷歌翻译